Evaluating Apps

Reviews principles and guidelines for evaluating mobile phone health applications. This will include components of usability, foundational theory, customer driven ratings, and the empirical evidence. Some specific applications for mood management and tobacco cessation are examined. This presentation was part of the retreat mini-symposium entitled: Smartphones, Sensors, and Social Networks: The New Tools of Health Behavior Change

Alcohol Use Disorder in the Age of Technology: A Review of Wearable Biosensors in Alcohol Use Disorder Treatment

Biosensors enable observation and understanding of latent physiological occurrences otherwise unknown or invasively detected. Wearable biosensors monitoring physiological constructs across a wide variety of mental and physical health conditions have become an important trend in innovative research methodologies. Within substance use research, explorations of biosensor technology commonly focus on identifying physiological indicators of intoxication to increase understanding of addiction etiology and to inform treatment recommendations. In this review, we examine the state of research in this area as it pertains to treatment of alcohol use disorders specifically highlighting the gaps in our current knowledge with recommendations for future research. Annually, alcohol use disorders affect approximately 15 million individuals. A primary focus of existing wearable technology-based research among people with alcohol use disorders is identifying alcohol intoxication. A large benefit of wearable biosensors for this purpose is they provide continuous readings in a passive manner compared with the gold standard measure of blood alcohol content (BAC) traditionally measured intermittently by breathalyzer or blood draw. There are two primary means of measuring intoxication with biosensors: gait and sweat. Gait changes have been measured via smart sensors placed on the wrist, in the shoe, and mobile device sensors in smart phones. Sweat measured by transdermal biosensors detects the presence of alcohol in the blood stream correlating to BAC. Transdermal biosensors have been designed in tattoos/skin patches, shirts, and most commonly, devices worn on the ankle or wrist. Transdermal devices were initially developed to help monitor court-ordered sobriety among offenders with alcohol use disorder. These devices now prove most useful in continuously tracking consumption throughout clinical trials for behavioral treatment modalities. More recent research has started exploring the uses for physical activity trackers and physiological arousal sensors to guide behavioral interventions for relapse prevention. While research has begun to demonstrate wearable devices\u27 utility in reducing alcohol consumption among individuals aiming to cutdown on their drinking, monitoring sustained abstinence in studies exploring contingency management for alcohol use disorders, and facilitating engagement in activity-based treatment interventions, their full potential to further aid in understanding of, and treatment for, alcohol use disorders has yet to be explored

Implementing At-the-Elbow Training in the Emergency Department: Feasibility, Outcomes, and Lessons

Introduction: Individuals who die by suicide often make ED contact the year prior to death. In accordance with Joint Commission recommendations, universal suicide screening has been implemented across UMass Memorial Health Care EDs, in an effort to improve quality of care for patients with suicide risk. We developed clinician decision aids, clinical workflows, and electronic health record (EHR) items, and introduced them using an \u27at-the-elbow\u27 training model in the ED. Methods: We programmed a data visualization application (Tableau), which pulls data directly from the EHR, to measure suicide-positive detection rates and physician secondary screener completion. Training logs were completed following every shift, and were used to identify barriers to training and lessons learned. Results: Over an average of 3.4 weeks per site, 207 ED clinicians (79 physicians, 32 residents, and approximately 96 nurses) were trained across all five EDs; encounters usually took 5-10 minutes. Trainers successfully reached 79 of 104 attending physicians (76%) across all sites. Suicide-detection rates increased post-training for three units, and secondary screening rates increased for four units. Three common barriers included understanding the definition of a suicide-positive screen; importance of physician secondary screening and how to use the tool for decision-making; and issues with nurse-physician communication of suicide risk. Discussion: The \u27at-the-elbow\u27 approach was an efficient way of reaching the majority of ED clinicians and facilitated rapid-cycle improvement in response to emerging barriers. However, this approach was resource-intensive for project staff and not sustainable long term. Integration into onboarding and a \u27train-the-trainer\u27 model could improve sustainability

Addressing Positive Suicide Screens in the Emergency Department: The Importance of Post-Discharge Follow-up

Suicide is the tenth leading cause of death and many of those who die by suicide have visited an emergency department (ED) in the months prior to their death. Thus, identification and treatment of suicidal ideation (SI) in the ED is essential to suicide prevention efforts. Although a recent effort to implement universal SI screening has identified more patients with suicide risk, there are still barriers to further risk assessment and intervention, including: patients being too ill, language differences, physician caseload, length of SI evaluation and intervention, staff availability and communication with emergency mental health (EMH) services, and stigma surrounding risk responsibility. To address these issues following the Zero Suicide Model, in November 2017 the pre-existing Behavioral Health Service (BHS) expanded their care to this population, improved communication with EMH to reduce patient burden, and implemented a follow-up call system to contact patients within 48 hours post-discharge. Since November, 61 patients were identified as not receiving further SI evaluation or resources while in the ED. Twenty-four (39.3%) of these patients were successfully contacted by phone, with 15 (62.5%) receiving resources and 9 (37.5%) declining resources due to existing services. All patients with available addresses (86.8%) were sent Caring Contact Cards with information on suicide hotlines and psychiatric emergency services. By attempting calls multiple times, mailing resources, and being brief, yet detailed when evaluating, more patients\u27 SI needs are being treated. The ultimate goal is to provide services to all patients who screen positive for suicide risk presenting to the ED

Emergency Department Super-utilizer Program Involvement: Pilot Data and Methods Challenges

Super-utilizers are patients who use extreme amounts of medical services, often due to comorbid medical, social, and mental health issues. The MyLink Evaluation Project (MEP) studies MyLink, a program that connects super-utilizers with community support workers (CSWs) to improve the patient experience and reduce costs. The MEP-eligible population is ≥18 years old with at least 5 Emergency Department (ED) visits within 12 months and no other exclusions (e.g., language barriers, living out-of-region). During MEP’s pilot, among 58 eligible patients, 28 consented to being referred to MyLink and followed up. Of these, 7 could not be located for follow-up, 8 refused enrollment, and the remaining 13 enrolled and “engaged” (had at least 3 face-to-face contacts and developed an initial plan). All 13 enrollees were followed at 6 months vs. 4 of the 8 not enrolled. Consequently, we expect about 50% of eligible patients to consent to the main randomized study, with the vast majority of the MyLink-assigned group becoming engaged and completing follow-up. Achieving this requires identifying patients in real-time at the ED, frequent communications between researchers and CSWs, cultivating rapport during patient referral, enrollment, and follow-up, coordinating with other care management programs serving our patients, and adhering to MEP protocols that are rapidly evolving to address and overcome barriers. Challenges include: increasingly heavy CSW case-loads that decrease “warm” handoffs during the ED visit; problematic patient contact information; and incomplete program and follow-up assessments due to patient withdrawal, relocation, or death. These challenges lead to missing quality-of-life and healthcare utilization data needed for program evaluation. To reduce incomplete assessments, we lengthened time windows and expanded outreach methods (e.g., in-person upon ED revisit, web and medical record searches for updated contact information). We hypothesize that MyLink will improve patient quality-of-life and reduce ED utilization and total costs of care for super-utilizers

Health Evaluation and Referral Assistant: A Randomized Controlled Trial of a Web-Based Screening, Brief Intervention, and Referral to Treatment System to Reduce Risky Alcohol Use Among Emergency Department Patients

BACKGROUND: Computer technologies hold promise for implementing alcohol screening, brief intervention, and referral to treatment (SBIRT). Questions concerning the most effective and appropriate SBIRT model remain. OBJECTIVE: The aim of this study was to evaluate the impact of a computerized SBIRT system called the Health Evaluation and Referral Assistant (HERA) on risky alcohol use treatment initiation. METHODS: Alcohol users (N=319) presenting to an emergency department (ED) were considered for enrollment. Those enrolled (n=212) were randomly assigned to the HERA, to complete a patient-administered assessment using a tablet computer, or a minimal-treatment control, and were followed for 3 months. Analyses compared alcohol treatment provider contact, treatment initiation, treatment completion, and alcohol use across condition using univariate comparisons, generalized estimating equations (GEEs), and post hoc chi-square analyses. RESULTS: HERA participants (n=212; control=115; intervention=97) did not differ between conditions on initial contact with an alcohol treatment provider, treatment initiation, treatment completion, or change in risky alcohol use behavior. Subanalyses indicated that HERA participants, who accepted a faxed referral, were more likely to initiate contact with a treatment provider and initiate treatment for risky alcohol use, but were not more likely to continue engaging in treatment, or to complete treatment and change risky alcohol use behavior over the 3-month period following the ED visit. CONCLUSIONS: The HERA promoted initial contact with an alcohol treatment provider and initiation of treatment for those who accepted the faxed referral, but it did not lead to reduced risky alcohol use behavior. Factors which may have limited the HERA\u27s impact include lack of support for the intervention by clinical staff, the low intensity of the brief and stand-alone design of the intervention, and barriers related to patient follow-through, (eg, a lack of transportation or childcare, fees for services, or schedule conflicts). TRIAL REGISTRATION: International Standard Randomized Controlled Trial Number (ISRCTN): NCT01153373; https://clinicaltrials.gov/ct2/show/NCT01153373 (Archived by WebCite at http://www.webcitation.org/6pHQEpuIF)

Computer Administered Safety Planning for Individuals at Risk for Suicide: Development and Usability Testing

BACKGROUND: Safety planning is a brief intervention that has become an accepted practice in many clinical settings to help prevent suicide. Even though it is quick compared to other approaches, it frequently requires 20 min or more to complete, which can impede adoption. A self-administered, Web-based safety planning application could potentially reduce clinician time, help promote standardization and quality, and provide enhanced ability to share the created plan. OBJECTIVE: The aim of this study was to design, build, and test the usability of a Web-based, self-administered safety planning application. METHODS: We employed a user-centered software design strategy led by a multidisciplinary team. The application was tested for usability with a target sample of suicidal patients. Detailed observations, structured usability ratings, and Think Aloud procedures were used. Suicidal ideation intensity and perceived ability to cope were assessed pre-post engagement with the Web application. RESULTS: A total of 30 participants were enrolled. Usability ratings were generally strong, and all patients successfully built a safety plan. However, the completeness of the safety plan varied. The mean number of steps completed was 5.5 (SD 0.9) out of 6, with 90% (27/30) of participants completing at least 5 steps and 67% (20/30) completing all 6 steps. Some safety planning steps were viewed as inapplicable to some individuals. Some confusion in instructions led to modifications to improve understandability of each step. Ratings of suicide intensity after completion of the application were significantly lower than preratings, pre: mean 5.11 (SD 2.9) versus post: mean 4.46 (SD 3.0), t27=2.49, P=.02. Ratings of ability to cope with suicidal thoughts after completion of the application were higher than preratings, with the difference approaching statistical significance, pre: mean 5.93 (SD 2.9), post: mean 6.64 (SD 2.4), t27=-2.03, P=.05. CONCLUSIONS: We have taken the first step toward identifying the components needed to maximize usability of a self-administered, Web-based safety planning application. Results support initial consideration of the application as an adjunct to clinical contact. This allows for the clinician or other personnel to provide clarification, when needed, to help the patient build the plan, and to help review and revise the draft

The Dynamic Assessment and Referral System for Substance Abuse (DARSSA): development, functionality, and end-user satisfaction

The Dynamic Assessment and Referral System for Substance Abuse (DARSSA) conducts a computerized substance abuse assessment; prints personalized summary reports that include tailored substance abuse treatment referral lists; and, for individuals who provide authorization, automatically faxes their contact information to a best match substance abuse treatment provider (dynamic referral). After piloting the program and resolving problems that were noted, we enrolled a sample of 85 medical patients. The DARSSA identified 48 (56%) participants who were risky substance users, many of whom had not been identified during their routine medical assessment. Mean satisfaction scores for all domains ranged between Good to Excellent across patients, nurses, doctors, and substance abuse treatment providers. The median completion time was 13min. Of the 48 risky substance using participants, 20 (42%) chose to receive a dynamic referral. The DARSSA provides a user-friendly, desirable service for patients and providers. It has the potential to improve identification of substance abuse in medical settings and to provide referrals that would not routinely be provided. Future studies are planned to establish its efficacy at promoting treatment initiation and abstinence

Patient preferences for emergency department-initiated tobacco interventions: a multicenter cross-sectional study of current smokers

BACKGROUND: The emergency department (ED) visit provides a great opportunity to initiate interventions for smoking cessation. However, little is known about ED patient preferences for receiving smoking cessation interventions or correlates of interest in tobacco counseling. METHODS: ED patients at 10 US medical centers were surveyed about preferences for hypothetical smoking cessation interventions and specific counseling styles. Multivariable linear regression determined correlates of receptivity to bedside counseling. RESULTS: Three hundred seventy-five patients were enrolled; 46% smoked at least one pack of cigarettes per day, and 11% had a smoking-related diagnosis. Most participants (75%) reported interest in at least one intervention. Medications were the most popular (e.g., nicotine replacement therapy, 54%), followed by linkages to hotlines or other outpatient counseling (33-42%), then counseling during the ED visit (33%). Counseling styles rated most favorably involved individualized feedback (54%), avoidance skill-building (53%), and emphasis on autonomy (53%). In univariable analysis, age (r = 0.09), gender (average Likert score = 2.75 for men, 2.42 for women), education (average Likert score = 2.92 for non-high school graduates, 2.44 for high school graduates), and presence of smoking-related symptoms (r = 0.10) were significant at the p \u3c 0.10 level and thus were retained for the final model. In multivariable linear regression, male gender, lower education, and smoking-related symptoms were independent correlates of increased receptivity to ED-based smoking counseling. CONCLUSIONS: In this multicenter study, smokers reported receptivity to ED-initiated interventions. However, there was variability in individual preferences for intervention type and counseling styles. To be effective in reducing smoking among its patients, the ED should offer a range of tobacco intervention options

Motivation rulers for smoking cessation: a prospective observational examination of construct and predictive validity

BACKGROUND: Although popular clinically, the psychometric properties of motivation rulers for tobacco cessation are unknown. This study examined the psychometric properties of rulers assessing importance, readiness, and confidence in tobacco cessation. METHODS: This observational study of current smokers was conducted at 10 US emergency departments (EDs). Subjects were assessed during their ED visit (baseline) and reassessed two weeks later. We examined intercorrelations between the rulers as well as their construct and predictive validity. Hierarchical multinomial logistic regressions were used to examine the rulers\u27 predictive ability after controlling for covariables. RESULTS: We enrolled 375 subjects. The correlations between the three rulers ranged from 0.50 (between Important and Confidence) to 0.70 (between Readiness and Confidence); all were significant (p \u3c 0.001). Individuals in the preparation stage displayed the highest motivation-ruler ratings (all rulers F 2, 363 \u3e/= 43; p \u3c 0.001). After adjusting for covariables, each of the rulers significantly improved prediction of smoking behavior change. The strength of their predictive ability was on par with that of stage of change. CONCLUSION: Our results provide preliminary support for the psychometric soundness of the importance, readiness, and confidence rulers